1. Background and Relevant Art
Computer systems and related technology affect many aspects of society. Indeed, the computer system's ability to process information has transformed the way we live and work. Computer systems now commonly perform a host of tasks (e.g., word processing, scheduling, accounting, etc.) that prior to the advent of the computer system were performed manually. More recently, computer systems have been coupled to one another and to other electronic devices to form both wired and wireless computer networks over which the computer systems and other electronic devices can transfer electronic data. Accordingly, the performance of many computing tasks are distributed across a number of different computer systems and/or a number of different computing environments.
In some environments, many remote users access applications and data through a centralized remote location. For example, a number of client machines can access a Web service via the Internet. In many environments, and especially for services (or other applications) with increased workload, multiple instances of the service (or application) are created to provide a better user experience. The multiple instances of the service can even be spread across a number of different computer systems, such as, for example, in a server farm.
Multiple computer system environments typically include some type of network load balancing technology to more evenly distribute workload between across instances of a service. However, many load balancing algorithms are somewhat primitive and typically do not consider message content when determining where to send a message. For example, some load balancing algorithms utilize simple round robin techniques.
Some more sophisticated load balancing algorithms attempt to insure that messages destined for the same instance of a service are delivered to the host where the service is currently loaded. Network load balancers can used different affinitization schemes, such as, session ID or client Internet Protocol (“IP”) address to attempt to deliver messages to the correct host. However, some types of services, such as, for example, services that used workflows, may cause these and similar affinitization schemes to perform in a less than optimal manner.
For example, the lifetime of a workflow can exceed the time of a client session, or have many distinct clients communicating with same workflow instance. Further, workflow services can use correlation to affinitize messages to service instances. However, the data used to generate correlation keys is typically contained in a message and can not be read by a network load balancer. Additionally, auxiliary services, such as, for example, a timer service, can move workflow instances between hosts in a scale out farm. As a result, there is often no way for a network load balancer to deterministically know the correct location of a workflow instance
Instead of using a separate load balancer, some systems utilize a queue that both stores messages and handles the distribution of messages to service instances. However in queue based systems, the queue is tightly coupled to the service instances. Thus, changes to the service can result in incompatibilities with the queue. Queue based systems also tend to have higher latency since the queue acts as a bottleneck to message distribution. Further, queue based systems make the use of transactions more difficult, since message delivery is tightly coupled to the queue. Additionally, queue based systems have various inefficiencies with respect to security.